Quantum-accessible reinforcement learning beyond strictly epochal environments

In recent years, quantum-enhanced machine learning has emerged as a particularly fruitful application of quantum algorithms, covering aspects of supervised, unsupervised and reinforcement learning. Reinforcement learning offers numerous options of how quantum theory can be applied, and is arguably the least explored, from a quantum perspective. Here, an agent explores an environment and tries to find a behavior optimizing some figure of merit. Some of the first approaches investigated settings where this exploration can be sped-up, by considering quantum analogs of classical environments, which can then be queried in superposition. If the environments have a strict periodic structure in time (i.e. are strictly episodic), such environments can be effectively converted to conventional oracles encountered in quantum information. However, in general environments, we obtain scenarios that generalize standard oracle tasks. In this work we consider one such generalization, where the environment is not strictly episodic, which is mapped to an oracle identification setting with a changing oracle. We analyze this case and show that standard amplitude-amplification techniques can, with minor modifications, still be applied to achieve quadratic speed-ups, and that this approach is optimal for certain settings. This results constitutes one of the first generalizations of quantum-accessible reinforcement learning.Comment: 8+9 pages, 2 figure

Noisy distributed sensing in the Bayesian regime

We consider non-local sensing of scalar signals with specific spatial dependence in the Bayesian regime. We design schemes that allow one to achieve optimal scaling and are immune to noise sources with a different spatial dependence than the signal. This is achieved by using a sensor array of spatially separated sensors and constructing a multi-dimensional decoherence free subspace. While in the Fisher regime with sharp prior and multiple measurements only the spectral range $\Delta$ is important, in single-shot sensing with broad prior the number of available energy levels $L$ is crucial. We study the influence of $L$ and $\Delta$ also in intermediate scenarios, and show that these quantities can be optimized separately in our setting. This provides us with a flexible scheme that can be adapted to different situations, and is by construction insensitive to given noise sources.Comment: 9 pages, 1 figur